Intelligent red eye removal

ABSTRACT

A device measures one or more of a distance from an eye of a subject, a blink of the eye, and lighting adjacent to the subject, calculates automated red eye removal settings based on one or more of the measured distance, blink, and lighting, automatically removes red eye from an image of the subject based on the calculated automated red eye removal settings, captures the image of the subject, and stores the captured image.

BACKGROUND

Images may be captured by a variety of consumer electronic devices, suchas mobile communication devices (e.g., cell phones, personal digitalassistants (PDAs), etc.), cameras (e.g., contemporary cameras or digitalcameras), video cameras, etc. Many times, captured images of eyes of asubject (e.g., a person, an animal, etc.) may be ruined due to red-eyeeffect (or “red eye”). The red-eye effect is the appearance of red eyesin images captured using a flash device or flash. Light from the flashoccurs too fast for the iris of the eye to close the pupil. Light ispassed through a blood-rich area (called the choroid) alongside the irisand then strikes the retina. Some of the light is reflected back outthrough the iris, and a captured image records this reflected red light,which has passed twice through the reddish choroid. The red-eye effectmay include color variations of the reflected light from species tospecies (e.g., cats may display blue, yellow, pink, or green eyes inimages captured with flash).

SUMMARY

According to one aspect, a method may include measuring one or more of adistance from an eye of a subject, a blink of the eye, and lightingadjacent to the subject, calculating automated red eye removal settingsbased on one or more of the measured distance, blink, and lighting,automatically removing red eye from an image of the subject based on thecalculated automated red eye removal settings, capturing the image ofthe subject, and storing the captured image.

Additionally, the method may include providing for display an automaticred eye removal option, and receiving selection of the automatic red eyeremoval option.

Additionally, the method may include determining manual red eye removalsuggestions based on the measured distance, blink, and lighting,providing for display a manual red eye removal option, receivingselection of the manual red eye removal option, providing for displaythe manual red eye removal suggestions, and capturing the image of thesubject when the manual red eye removal suggestions remove red eye.

Additionally, the method may include providing for display a suggestionto increase ambient lighting or providing for display a suggestion tomove toward or away from the subject.

Additionally, the method may include automatically removing red eye fromthe subject image by increasing or decreasing a length of time of apre-flash, automatically removing red eye from the subject image bydelaying a flash of a device until the subject looks away from thedevice, automatically removing red eye from the subject image bydirecting light from the flash at an oblique angle to the eye,automatically removing red eye from the subject image by opening a lensaperture and omitting the flash, or automatically removing red eye fromthe subject image by reducing a shutter speed and omitting the flash.

According to another aspect, a user device may include a distancemonitoring device that measures a distance from an eye of a subject, ablink monitoring device that measures a blink of the eye, a lightingmonitoring device that measures lighting adjacent to the subject, andprocessing logic configured to calculate automated red eye removalsettings based on the measured distance, blink, and lighting,automatically remove red eye from an image of the subject based on thecalculated automated red eye removal settings, and capture the subjectimage.

Additionally, the distance monitoring device may include one of aproximity sensor, a laser distance sensor, a distance sensor using echolocation with high frequency sound waves, or an infrared distancesensor.

Additionally, the blink monitoring device may include one of an eyegazing sensor, an infrared camera and infrared illuminator combination,or a device used to sense, locate, and follow the movement of the eye.

Additionally, the blink monitoring device may detect the blink as a dualsymmetrical difference in images of the subject.

Additionally, the lighting monitoring device may include one of anambient light sensor, a charge coupled device (CCD) light sensor, or acomplementary metal oxide semiconductor (CMOS) light sensor.

Additionally, the processing logic may be configured to provide fordisplay an automatic red eye removal option, and receive selection ofthe automatic red eye removal option.

Additionally, the processing logic may be configured to provide fordisplay the subject image with red eyes prior to the automatic removalof red eye, provide for display the subject image with non-red eyesafter the automatic removal of red eye, provide for display anindication of removal of red eye, and provide for display instructionsfor capturing the subject image.

Additionally, the processing logic may be configured to determine manualred eye removal suggestions based on the measured distance, blink, andlighting, provide for display a manual red eye removal option, receiveselection of the manual red eye removal option, provide for display themanual red eye removal suggestions, and automatically capture thesubject image when the manual red eye removal suggestions remove redeye.

Additionally, the processing logic may be configured to at least one ofprovide for display a suggestion to increase ambient lighting, orprovide for display a suggestion to move toward or away from thesubject.

Additionally, the processing logic may be configured to provide fordisplay the subject image with red eyes prior to the manual removal ofred eye, provide for display the subject image with non-red eyes whenthe manual red eye removal suggestions remove red eye, provide fordisplay an indication of removal of red eye, and provide for displayinstructions for capturing the subject image.

Additionally, the processing logic may be configured to at least one ofautomatically remove red eye from the subject image by increasing ordecreasing a length of time of a pre-flash, automatically remove red eyefrom the subject image by delaying a flash until the subject looks awayfrom the user device, automatically remove red eye from the subjectimage by directing light from the flash at an oblique angle to the eye,automatically remove red eye from the subject image by opening a lensaperture and omitting the flash, or automatically remove red eye fromthe subject image by reducing a shutter speed and omitting the flash.

Additionally, the user device may include one of a mobile communicationdevice, a laptop, a personal computer, a camera, a video camera,binoculars with a camera, or a telescope with a camera.

According to yet another aspect, a user device may include one or moredevices configured to measure a distance from an eye of a subject whoseimage is to be captured, measure a blink of the eye, and measurelighting adjacent to the subject. The user device may also includeprocessing logic configured to provide for display an automatic red eyeremoval option and a manual red eye removal option, receive selection ofthe automatic red eye removal option or the manual red eye removaloption, calculate automated red eye removal settings based on themeasured distance, blink, and lighting when the automatic red eyeremoval option is selected, automatically remove red eye from an imageof the subject based on the calculated automated red eye removalsettings when the automatic red eye removal option is selected,determine manual red eye removal suggestions based on the measureddistance, blink, and lighting when the manual red eye removal option isselected, provide for display the manual red eye removal suggestionswhen the manual red eye removal option is selected, and enable thesubject image to be captured when red eye is removed.

Additionally, the processing logic may be configured to provide fordisplay the subject image with red eyes prior to the removal of red eye,provide for display the subject image with non-red eyes when red eye isremoved, provide for display an indication of removal of red eye, andprovide for display instructions for capturing the subject image.

According to a further aspect, a system may include means for measuringa distance from an eye of a subject, means for measuring a blink of theeye, means for measuring lighting adjacent to the subject, means forcalculating automated red eye removal settings based on the measureddistance, blink, and lighting, means for automatically removing red eyefrom an image of the subject based on the calculated automated red eyeremoval settings, means for capturing the image of the subject, andmeans for storing the captured subject image.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate one or more implementationsdescribed herein and, together with the description, explain theseimplementations. In the drawings:

FIG. 1 is an exemplary diagram illustrating a configuration according toconcepts described herein;

FIG. 2 depicts an exemplary diagram of a user device, illustrated inFIG. 1;

FIG. 3 illustrates a diagram of exemplary components of the user devicedepicted in FIGS. 1 and 2;

FIG. 4 depicts a diagram of exemplary monitoring devices andinteractions with processing logic of the user device illustrated inFIGS. 1 and 2;

FIGS. 5-7 illustrate exemplary user interfaces capable of being providedby the user device depicted in FIGS. 1 and 2; and

FIGS. 8A-10 depict flow charts of an exemplary process for intelligentlyremoving red eye according to implementations described herein.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements. Also, the following detailed description does notlimit the invention.

Overview

Implementations described herein may predict red eye by detecting eyesof a subject (e.g., whose image is to be captured) using eye blinks, andmay eliminate red eye by increasing light sensitivity or by adapting apre-flash length of time based on a risk of red eye. For example, in oneimplementation, a user device may measure a distance from an eye of thesubject, may measure a blink of the subject's eye, and may measurelighting adjacent to the subject. The user device may also calculateautomated red eye removal settings based on the measured information,may determine manual red eye removal suggestions based on the measuredinformation, and may provide for display an automatic red eye removaloption and/or a manual red eye removal option. If the user devicereceives selection of the automatic red eye removal option, the userdevice may automatically remove red eye based on the calculatedautomated red eye removal settings, and may capture an image of thesubject. If the user device receives selection of the manual red eyeremoval option, the user device may provide for display the manual redeye removal suggestions, and may capture the image of the subject whenthe manual suggestions remove red eye.

Exemplary Configuration

FIG. 1 is an exemplary diagram illustrating a configuration 100according to concepts described herein. As illustrated, configurationmay include a user device 110 and an eye 120 of a subject whose image isto be captured by user device 110.

User device 110 may include any device capable of capturing an image.For example, user device 110 may include a mobile communication device(e.g., a radiotelephone, a personal communications system (PCS) terminalthat may combine a cellular radiotelephone with data processing, afacsimile, and data communications capabilities, a personal digitalassistant (PDA) that can include a radiotelephone, pager,Internet/intranet access, web browser, organizer, camera, a Dopplerreceiver, and/or global positioning system (GPS) receiver, a GPS device,a telephone, a cellular phone, etc.); a laptop; a personal computer; aprinter; a facsimile machine; a pager; a camera (e.g., a contemporarycamera or a digital camera); a video camera (e.g., a camcorder); acalculator; binoculars; a telescope; a GPS device; a gaming device; anyother device capable of utilizing a camera; a thread or process runningon one of these devices; and/or an object executable by one of thesedevices.

As used herein, a “camera” may include a device that may capture andstore images and/or video. For example, a digital camera may be anelectronic device that may capture and store images and/or videoelectronically instead of using photographic film as in contemporarycameras. A digital camera may be multifunctional, with some devicescapable of recording sound and/or video, as well as images.

Eye 120 may include one or more eyes of a subject (e.g., a person, ananimal, etc.) whose image is to be captured by user device 110. In oneimplementation, for example, eye 120 may produce the red-eye effect ifuser device 110 attempts to capture an image of the subject using aflash. Flash may include a device (e.g., an electronic flash unit) thatproduces an instantaneous flash of artificial light (e.g., around 1/3000of a second) at a color temperature (e.g., about 5500K) to helpilluminate a scene, capture quickly moving objects, create a differenttemperature light than ambient light, etc.

As further shown in FIG. 1, user device 110 may generate a distancesensing signal 130 that may be used to detect a distance between userdevice 110 and eye 120. A detected signal may be returned to user device110 as distance information 140 (e.g., a distance between user device110 and eye 120). User device 110 may also generate a blink detectionsignal 150 that may be used to detect a blink of eye 120. A detectedsignal may be returned to user device 110 as blink information 160.Blink information 160 may represent one or more blinks (e.g., closingsand openings) of eye 120, and pupil information (e.g., an abnormalcircular shape where the one or more blinks are detected, a contractileaperture in the iris of eye 120, etc.) associated with eye 120.Depending on distance information 140, blink information 160 may providean indication of a risk for red eye. User device 110 may generate alighting sensing signal 170 that may be used to detect lighting in anarea adjacent to and/or at eye 120. A detected signal may be returned touser device 110 as lighting information 180 (e.g., ambient lighting inan area adjacent to and/or at the subject). Further details of userdevice 110 are provided below in connection with FIGS. 2-7.

Although FIG. 1 shows exemplary elements of configuration 100, in otherimplementations, configuration 100 may contain fewer, different, oradditional elements than depicted in FIG. 1.

Exemplary User Device Configuration

FIG. 2 is an exemplary diagram of user device 110. As illustrated, userdevice 110 may include a housing 200, a speaker 210, a display 220,control buttons 230, a keypad 240, a microphone 250, and/or a camera260. Housing 200 may protect the components of user device 110 fromoutside elements. Speaker 210 may provide audible information to a userof user device 110.

Display 220 may provide visual information to the user. For example,display 220 may display text input into user device 110, text, images,video, and/or graphics received from another device, and/or informationregarding incoming or outgoing calls or text messages, emails, media,games, phone books, address books, the current time, etc. In oneexemplary implementation, display 220 may provide act as a viewfinderthat may aid user device 110 in capturing and/or storing video and/orimages. Control buttons 230 may permit the user to interact with userdevice 110 to cause user device 110 to perform one or more operations.For example, control buttons 230 may be used to cause user device 110 totransmit information. Keypad 240 may include a standard telephonekeypad. Microphone 250 may receive audible information from the user.Camera 260 may be provided on a back side of user device 110, and mayenable user device 110 to capture and/or store video and/or images(e.g., pictures).

Although FIG. 2 shows exemplary components of user device 110, in otherimplementations, user device 110 may contain fewer, different, oradditional components than depicted in FIG. 2. In still otherimplementations, one or more components of user device 110 may performone or more other tasks described as being performed by one or moreother components of user device 110.

FIG. 3 is a diagram of exemplary components of user device 110. Asillustrated, user device 110 may include processing logic 310, memory320, a user interface 330, a communication interface 340, an antennaassembly 350, and/or one or more monitoring devices 360.

Processing logic 310 may include a processor, a microprocessor, anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), or the like. Processing logic 310 may controloperation of user device 110 and its components. In one implementation,processing logic 310 may control operation of components of user device110 in a manner described herein.

Memory 320 may include a random access memory (RAM), a read-only memory(ROM), and/or another type of memory to store data and instructions thatmay be used by processing logic 310.

User interface 330 may include mechanisms for inputting information touser device 110 and/or for outputting information from user device 110.Examples of input and output mechanisms might include buttons (e.g.,control buttons 230, keys of keypad 240, a joystick, etc.) or a touchscreen interface to permit data and control commands to be input intouser device 110; a speaker (e.g., speaker 210) to receive electricalsignals and output audio signals; a microphone (e.g., microphone 250) toreceive audio signals and output electrical signals; a display (e.g.,display 220) to output visual information (e.g., text input into userdevice 110); a vibrator to cause user device 110 to vibrate; and/or acamera (e.g., camera 260) to receive video and/or images.

Communication interface 340 may include, for example, a transmitter thatmay convert baseband signals from processing logic 310 to radiofrequency (RF) signals and/or a receiver that may convert RF signals tobaseband signals. Alternatively, communication interface 340 may includea transceiver to perform functions of both a transmitter and a receiver.Communication interface 340 may connect to antenna assembly 350 fortransmission and/or reception of the RF signals.

Antenna assembly 350 may include one or more antennas to transmit and/orreceive RF signals over the air. Antenna assembly 350 may, for example,receive RF signals from communication interface 340 and transmit themover the air, and receive RF signals over the air and provide them tocommunication interface 340. In one implementation, for example,communication interface 340 may communicate with a network and/ordevices connected to a network.

Monitoring devices 360 may include any device capable of monitoringconditions associated with a subject (e.g., eye 120 of the subject)whose image is to be captured by user device 110. For example, in oneimplementation, monitoring devices 360 may include a distance monitoringdevice (e.g., a proximity sensor, a laser distance sensor, etc.), ablink monitoring device (e.g., an eye gazing sensor, an infrared cameraand infrared illuminator combination, etc.), and/or a lightingmonitoring device (e.g., an ambient light sensor, a charge coupleddevice (CCD) light sensor, a complementary metal oxide semiconductor(CMOS) light sensor, etc.). Further details of monitoring devices 360are provided below in connection with FIG. 4.

As will be described in detail below, user device 110 may performcertain operations described herein in response to processing logic 310executing software instructions of an application contained in acomputer-readable medium, such as memory 320. A computer-readable mediummay be defined as a physical or logical memory device. The softwareinstructions may be read into memory 320 from another computer-readablemedium or from another device via communication interface 340. Thesoftware instructions contained in memory 320 may cause processing logic310 to perform processes that will be described later. Alternatively,hardwired circuitry may be used in place of or in combination withsoftware instructions to implement processes described herein. Thus,implementations described herein are not limited to any specificcombination of hardware circuitry and software.

Although FIG. 3 shows exemplary components of user device 110, in otherimplementations, user device 110 may contain fewer, different, oradditional components than depicted in FIG. 3. In still otherimplementations, one or more components of user device 110 may performone or more other tasks described as being performed by one or moreother components of user device 110.

Exemplary Monitoring Devices/Interactions with Processing Logic

FIG. 4 is an exemplary diagram 400 of monitoring devices 360 andinteractions with processing logic 310. As illustrated, monitoringdevices 360 may include a distance monitoring device 410, a blinkmonitoring device 420, and/or a lighting monitoring device 430, andprocessing logic 310 may include red eye removal logic 440.

Distance monitoring device 410 may include any device capable ofmeasuring a distance between user device 110 and a subject (e.g. eye 120of the subject) whose image is to be captured by user device 110. Forexample, in one implementation, distance monitoring device 410 mayinclude a proximity sensor, a laser distance sensor, a distance sensorusing echo location with high frequency sound waves, an infrareddistance sensor, other distance sensors capable of measuring thedistance to the eye of the subject, etc. In other implementations,distance monitoring device 410 may include other components of userdevice 110 that are capable of measuring distance, such as processinglogic 310. As further shown in FIG. 4, distance monitoring device 410may generate distance sensing signal 130, and may receive distanceinformation 140 in response to distance sensing signal 130. Distancemonitoring device 410 may provide distance information 140 to processinglogic 310 (e.g., to red eye removal logic 440).

Blink monitoring device 420 may include any device capable of measuringone or more blinks of an eye (e.g., eye 120) or eyes of the subject. Forexample, in one implementation, blink monitoring device 420 may includean eye gazing sensor, an infrared camera and infrared illuminator (e.g.,a laser emitting diode) combination, a device used to sense, locate, andfollow the movement of an eye(s), any other sensor capable of detectingeye blinks, etc. For example, in one implementation, user device 110(e.g., a camera) may focus on one or both eyes of the subject and mayrecord blinks as the subject focuses upon user device 110. Contrast maybe used to locate the center of the pupil, infrared beams may be used tocreate a corneal reflection, and the triangulation of both may be usedto determine the blinks of the subject's eye(s). In otherimplementations, blink monitoring device 420 may include othercomponents of user device 110 that are capable of measuring blinks, suchas processing logic 310.

In still other implementations, user device 110 (e.g., blink monitoringdevice 420) may acquire images of the subject (e.g., via display 220).As each image is acquired, blink monitoring device 420 may subtract aprevious image to obtain a difference image. The resulting differenceimage may include a small boundary region around the outside of thesubject's head. If the subject's eyes happen to be closed in one of thetwo images, there may also be two small round regions over the eyeswhere a difference may be significant. Blink monitoring device 420 maythreshold (i.e., segment) the difference image, and may execute aconnected components algorithm on the threshold image. Blink monitoringdevice 420 may calculate a bounding box for each connected component. Acandidate for an eye may include a bounding box within a particularhorizontal and vertical size. Blink monitoring device 420 may detect twosuch candidates with a horizontal separation of a certain range ofsizes, and a small vertical difference in vertical separation. If blinkmonitoring device 420 detects this configuration of two small boundingboxes, a pair of blinking eyes may be determined (i.e., a blink may bedetected as a dual symmetrical (e.g., elliptical) difference in theimages). Such blink monitoring device 420 may be used in dark conditionsto find a blink because it looks for blinks rather than facial features.

As further shown in FIG. 4, blink monitoring device 420 may generateblink detection signal 150, and may receive blink information 160 inresponse to blink detection signal 150. Blink monitoring device 420 mayprovide blink information 160 to processing logic 310 (e.g., to red eyeremoval logic 440).

Lighting monitoring device 430 may include any device capable ofmeasuring lighting conditions associated with the subject and/or theeyes of the subject. For example, in one implementation, lightingmonitoring device 430 may include an ambient light sensor, a CCD lightsensor, a CMOS light sensor, etc. In other implementations, lightingmonitoring device 430 may include other components of user device 110that are capable of measuring lighting conditions, such as processinglogic 310. As further shown in FIG. 4, lighting monitoring device 430may generate lighting sensing signal 170, and may receive lightinginformation 180 in response to lighting sensing signal 170. Lightingmonitoring device 430 may provide lighting information 180 to processinglogic 310 (e.g., to red eye removal logic 440).

Red eye removal logic 440 may include any hardware and/or software basedlogic (e.g., processing logic 310) that enables user device 110 toremove red eye from an image to be captured by user device 110. In oneimplementation, red eye removal logic 440 may utilize distanceinformation 140, blink information 160, and/or lighting information 180to automatically or manually remove red eye from an image to be capturedby user device 110. In one example, as shown in FIG. 4, red eye removallogic 440 may utilize distance information 140, blink information 160,and/or lighting information 180 to automatically calculate pre-flashinformation 450. User device 110 may use pre-flash information 450 toautomatically remove red eye in an image by intelligently increasing ordecreasing a length of time of a pre-flash (e.g., depending on if redeye disappears). In another example, red eye removal logic 440 mayutilize distance information 140, blink information 160, and/or lightinginformation 180 to automatically calculate flash information 460. Userdevice 110 may use flash information 460 to automatically remove red eyein an image by detecting a gaze direction of the subject and delaying afiring of a flash until the subject looks away, by redirecting the flashaway from an optical axis of user device 110 so that light from theflash strikes the eyes at an oblique angle (e.g., the light may enterthe eyes in a direction away from the optical axis of user device 110,and may be refocused by the eyes' lenses back along the same axis, whichmay hide the retinas from user device 110), etc.

In a further example, red eye removal logic 440 may utilize distanceinformation 140, blink information 160, and/or lighting information 180to determine lighting and/or position information 470. User device 110may use lighting/position information 470 to remove red eye in an imageby instructing a user of user device 110 to manually increase lightingaround the subject and/or to manually move user device 110 toward oraway from the subject. In still a further example, red eye removal logic440 may utilize distance information 140, blink information 160, and/orlighting information 180 to automatically calculate other red eyeremoval information 480. User device 110 may use other red eye removalinformation 480 to automatically remove red eye in an image by capturingthe image without a flash, by opening a lens aperture of user device110, by reducing shutter speed of user device 110, by using a fasterfilm (e.g., if user device 110 is a contemporary camera), by pushingfilm development to increase an apparent film speed (e.g., if userdevice 110 is a contemporary camera), etc.

In one implementation, red eye removal logic 440 may utilize distanceinformation 140, blink information 160, and/or lighting information 180to automatically remove red eye without any input from the user. Forexample, red eye removal logic 440 may automatically select a bestmethod for removing red eye (e.g., if the subject's head is moving, themay delay firing of the flash until the angle is better; if it is fairlylight, then may use a larger aperture to avoid using flash, etc.), andmay automatically remove red eye using the selected method.

Although FIG. 4 shows exemplary components of processing logic 310 andmonitoring devices 360, in other implementations, processing logic 310and monitoring devices 360 may contain fewer, different, or additionalcomponents than depicted in FIG. 4. In still other implementations, oneor more components of processing logic 310 and monitoring devices 360may perform one or more other tasks described as being performed by oneor more other components of processing logic 310 and monitoring devices360.

Exemplary User Interfaces

FIGS. 5-7 depict exemplary user interfaces that may be provided by userdevice 110 (e.g., via user interface 330 and display 220). A userinterface 500, as illustrated FIG. 5, may be provided on display 220 ofuser device 110 if a user aligns a subject with a viewfinder of userdevice 110. User interface 500 may include a subject image 510, red eyes520, an automatically remove red eye option 530, and a manually removered eye option 540.

Subject image 510 may include an image of the subject captured by aviewfinder of user device 110. In one implementation, for example,subject image 510 may include an image of person with red eyes 520(e.g., experiencing the red-eye effect). If user device 110 capturessubject image 510, at this time, the captured image may include unwantedred eyes 520.

Red eyes 520 may be provided by user device 110 in subject image 510(e.g., where the eyes of the subject image may be determined to belocated) if there is a risk of a captured image experiencing the red-eyeeffect. In one implementation, for example, red eyes 520 may include aneye-shaped image that includes a variety of colors (e.g., red, blue,green, etc.).

Automatically remove red eye option 530 may include a mechanism (e.g.,an icon, a button, a link, etc.) that may be selected by a user of userdevice 110. If the user selects automatically remove red eye option 530,user device 110 (e.g., red eye removal logic 440) may receive theselection, may automatically calculate automated red eye removalsettings (e.g., based on the measured distance information 140, blinkinformation 160, and/or lighting information 180), and may automaticallyremove red eye from subject image 510 based on the calculated automatedred eye removal settings, as described above in connection with FIG. 4.

Manually remove red eye option 540 may include a mechanism (e.g., anicon, a button, a link, etc.) that may be selected by a user of userdevice 110. If the user selects manually remove red eye option 540, userdevice 110 (e.g., red eye removal logic 440), may receive the selection,may determine red eye removal suggestions (e.g., based on the measureddistance information 140, blink information 160, and/or lightinginformation 180), and may provide for display the manual red eye removalsuggestions (e.g., via display 220), as described below in connectionwith FIG. 7.

If the user selects automatically remove red eye option 530, a userinterface 600, as shown in FIG. 6, may be provided on display 220 ofuser device 110. User device 110 may have calculated the automated redeye removal settings (e.g., based on the measured distance information140, blink information 160, and/or lighting information 180) and mayhave automatically removed red eye from subject image 510 based on thecalculated automated red eye removal settings. Thus, user interface 600may include non-red eyes 610, an indication 620 that red eye has beenremoved, and a capture image option 630.

Non-red eyes 610 may be provided by user device 110 in subject image 510(e.g., where the eyes of the subject image may be determined to belocated) if there is no risk of a captured image experiencing thered-eye effect. In one implementation, for example, non-red eyes 610 mayinclude an eye-shaped image that includes a variety of colors other thanred (e.g., black).

Indication 620 may provide a visual indication (e.g., textual,graphical, textual/graphical, etc.) that red eye has been removed fromsubject image 510. For example, as shown in FIG. 6, indication 620 maystate “RED EYE REMOVED.” This may provide an indication that subjectimage 510 is ready to be captured by user device 110.

Capture image option 630 may include a mechanism (e.g., an icon, abutton, a link, etc.) that may be selected by a user of user device 110.If the user selects capture image option 630, user device 110 maycapture subject image 510 and/or may store the captured subject image510 (e.g., in memory 320). The captured subject image 510 may notinclude red eye.

If the user selects manually remove red eye option 540, a user interface700, as shown in FIG. 7, may be provided on display 220 of user device110. User device 110 may have determined the manual red eye removalsuggestions (e.g., based on the measured distance information 140, blinkinformation 160, and/or lighting information 180) but red eyes 520 (notshown in FIG. 7) may remain in subject image 510 until one or more ofthe manual red eye removal suggestions are implemented by the user. Asillustrated, user interface 700 may include manual red eye removalsuggestions, such as an increase lighting suggestion 710 and a movetoward/away from subject suggestion 720, non-red eyes 730, an indication740 that red eye has been removed, and a capture image option 750.Non-red eyes 730, indication 740, and/or capture image option 750 maynot be initially displayed by user interface 700 until the userimplements one or more of the manual red eye removal suggestions (e.g.,increases the lighting around the subject) and red eye is removed. Ifone or more of the manual red eye removal suggestions eliminates red eyein subject image 510, non-red eyes 730, indication 740, and/or captureimage option 750 may be displayed by user interface 700.

Lighting suggestion 710 may provide a visual suggestion (e.g., textual,graphical, textual/graphical, etc.) to increase lighting around,adjacent to, etc. the subject. If the lighting is increased, user device110 may detect that lighting is sufficient to omit using a flash, whichmay eliminate red eye in subject image 510.

Move toward/away from subject 720 may provide a visual suggestion (e.g.,textual, graphical, textual/graphical, etc.) to move user device 110toward or away from the subject. Movement of user device 110 toward oraway from the subject may omit the need for the flash, which mayeliminate red eye in subject image 510.

Non-red eyes 730 may be provided by user device 110 in subject image 510(e.g., where the eyes of the subject image may be determined to belocated) if there is no risk of a captured image experiencing thered-eye effect. In one implementation, for example, non-red eyes 730 mayinclude an eye-shaped image that includes a variety of colors other thanred (e.g., black).

Indication 740 may provide a visual indication (e.g., textual,graphical, textual/graphical, etc.) that may be displayed when themanual red eye removal suggestions have eliminated red eye, and mayindicate that red eye has been removed from subject image 510. Forexample, as shown in FIG. 7, indication 740 may state “RED EYE REMOVED.”This may provide an indication that subject image 510 is ready to becaptured by user device 110.

Capture image option 750 may include a mechanism (e.g., an icon, abutton, a link, etc.) that may be displayed when the manual red eyeremoval suggestions have eliminated red eye, and may be selected by auser of user device 110. If the user selects capture image option 750,user device 110 may capture subject image 510 and/or may store thecaptured subject image 510 (e.g., in memory 320). The captured subjectimage 510 may not include red eye.

In one implementation, red eyes may automatically be previewed in aviewfinder (e.g., display 220) of user device 110 in real time byplacing red eye graphics (e.g., dots; graphics of red eye pupils,appropriate for the subject image, that may be selected from a libraryof red eye graphics; etc.) in a position where the eyes of the subjectimage are located. The red eye graphics may be motion locked to thesubject's face (e.g., by motion and/or face tracking) so that the redeye graphics move with the subject. This may provide feedback as towhether red eye is likely, and whether red eye may disappear (e.g.,disappear in the viewfinder) because the subject moved (e.g., thesubject moved his/her head) or lighting conditions changed (e.g.,ambient light is decreased).

Although FIGS. 5-7 show exemplary elements of user interfaces 500-700,in other implementations, user interfaces 500-700 may contain fewer,different, or additional elements than depicted in FIGS. 5-7.

Exemplary Process

FIGS. 8A-10 depict flow charts of an exemplary process 800 forintelligently removing red eye according to implementations describedherein. In one implementation, process 800 may be performed by hardwareand/or software components of user device 110 (e.g., processing logic310 and/or red eye removal logic 440). In other implementations, process800 may be performed by hardware and/or software components of userdevice 110 (e.g., processing logic 310 and/or red eye removal logic 440)in combination with hardware and/or software components of anotherdevice (e.g., communicating with user device 110 via communicationinterface 340).

As illustrated in FIG. 8A, process 800 may begin with a measurement of adistance from an eye of a subject (block 805), a measurement of a blinkof the eye (block 810), and a measurement of lighting adjacent to thesubject (block 815). For example, in one implementation described abovein connection with FIG. 4, distance monitoring device 410 of user device110 may generate distance sensing signal 130, and may receive distanceinformation 140 in response to distance sensing signal 130. Distancemonitoring device 410 may provide distance information 140 to processinglogic 310 (e.g., to red eye removal logic 440) of user device 110. Blinkmonitoring device 420 of user device 110 may generate blink detectionsignal 150, and may receive blink information 160 in response to blinkdetection signal 150. Blink monitoring device 420 may provide blinkinformation 160 to processing logic 310 (e.g., to red eye removal logic440). Lighting monitoring device 430 may generate lighting sensingsignal 170, and may receive lighting information 180 in response tolighting sensing signal 170. Lighting monitoring device 430 may providelighting information 180 to processing logic 310 (e.g., to red eyeremoval logic 440).

Returning to FIG. 8A, automated red removal settings may be calculatedbased on the measured information (block 820), and manual red eyeremoval suggestions may be determined based the measured information(block 825). For example, in one implementation described above inconnection with FIG. 4, red eye removal logic 440 may utilize distanceinformation 140, blink information 160, and/or lighting information 180to automatically or manually remove red eye from an image to be capturedby user device 110. In one example, red eye removal logic 440 mayutilize distance information 140, blink information 160, and/or lightinginformation 180 to automatically calculate pre-flash information 450,which may automatically remove red eye in an image by intelligentlyincreasing or decreasing a length of time of a pre-flash (e.g.,depending on if red eye disappears). In another example, red eye removallogic 440 may utilize distance information 140, blink information 160,and/or lighting information 180 to automatically calculate flashinformation 460, which may automatically remove red eye in an image bydetecting a gaze direction of the subject and delaying a firing of aflash until the subject looks away, by redirecting the flash away froman optical axis of user device 110 so that light from the flash strikesthe eyes at an oblique angle. In a further example, red eye removallogic 440 may utilize distance information 140, blink information 160,and/or lighting information 180 to determine lighting and/or positioninformation 470, which may remove red eye in an image by instructing auser of user device 110 to manually increase lighting around the subjectand/or to manually move user device 110 toward or away from the subject.

As further shown in FIG. 8A, an automatic red eye removal option and amanual red eye removal option may be provided for display (block 830).For example, in one implementation described above in connection withFIG. 5, user interface 500 or user device 110 may display automaticallyremove red eye option 530 and manually remove red eye option 540.Automatically remove red eye option 530 may include a mechanism (e.g.,an icon, a button, a link, etc.) that may be selected by a user of userdevice 110. Manually remove red eye option 540 may include a mechanism(e.g., an icon, a button, a link, etc.) that may be selected by a userof user device 110. In other implementations, user device 110 mayprovide the user a single option (i.e., automatically remove red eyeoption 530 or manually remove red eye option 540).

If a user of user device 110 selects the displayed automatic red eyeremoval option (block 830), the process blocks depicted in FIG. 8B maybe implemented. As illustrated, a selection of the automatic red eyeremoval option may be received (block 835), and red eye may beautomatically removed based on the calculated automated red eye removalsettings (block 840). For example, in one implementation described abovein connection with FIG. 5, if the user selects automatically remove redeye option 530, user device 110 (e.g., red eye removal logic 440) mayreceive the selection, and may automatically remove red eye from subjectimage 510 based on the calculated automated red eye removal settings.

Returning to FIG. 8B, an image of the subject may be captured (block845). For example, in one implementation described above in connectionwith FIG. 6, user interface 600 may be provided on display 220 of userdevice 110 and may include capture image option 630. Capture imageoption 630 may include a mechanism (e.g., an icon, a button, a link,etc.) that may be selected by a user of user device 110. If the userselects capture image option 630, user device 110 may capture subjectimage 510 and/or may store the captured subject image 510 (e.g., inmemory 320). Alternatively, user device 110 may capture the imageautomatically in response to the selecting automatically remove red eyeoption 530. In either event, the captured subject image 510 may notinclude red eye.

If a user of user device 110 selects the displayed manually red eyeremoval option (block 830), the process blocks in FIG. 8C may beimplemented. As illustrated, a selection of the manual red eye removaloption may be received (block 850), and the manual red eye removalsuggestions may be provided for display (block 855). For example, in oneimplementation described above in connection with FIGS. 5 and 7, if theuser selects manually remove red eye option 540, user device 110 (e.g.,red eye removal logic 440) may receive the selection, and may providefor display the manual red eye removal suggestions (e.g., via userinterface 700 on display 220). User interface 700 may include manual redeye removal suggestions, such as increase lighting suggestion 710 andmove toward/away from subject suggestion 720. Lighting suggestion 710may provide a visual suggestion (e.g., textual, graphical,textual/graphical, etc.) to increase lighting around, adjacent to, etc.the subject. If the lighting is increased, user device 110 may detectthat lighting is sufficient to omit using a flash, which may eliminatered eye in subject image 510. Move toward/away from subject 720 mayprovide a visual suggestion (e.g., textual, graphical,textual/graphical, etc.) to move user device 110 toward or away from thesubject. Movement of user device 110 toward or away from the subject mayeliminate red eye in subject image 510.

Returning to FIG. 8C, an image of the subject may be captured when themanual suggestions remove red eye (block 860). For example, in oneimplementation described above in connection with FIG. 7, user interface700 may be provided on display 220 of user device 110 and may includecapture image option 750. Capture image option 750 may include amechanism (e.g., an icon, a button, a link, etc.) that may be displayedwhen the manual red eye removal suggestions have eliminated red eye, andmay be selected by a user of user device 110. If the user selectscapture image option 750, user device 110 may capture subject image 510and/or may store the captured subject image 510 (e.g., in memory 320).The captured subject image 510 may not include red eye.

Process block 840 may include the process blocks depicted in FIG. 9. Asillustrated, process block 840 may include automatically removing redeye by increasing or decreasing a length of time of a pre-flash (block900), and/or automatically removing red eye by delaying a flash untilthe subject looks away (block 910). For example, in one implementationdescribed above in connection with FIG. 4, red eye removal logic 440 mayutilize distance information 140, blink information 160, and/or lightinginformation 180 to automatically calculate pre-flash information 450,which may automatically remove red eye in an image by intelligentlyincreasing or decreasing a length of time of a pre-flash (e.g.,depending on if red eye disappears). Red eye removal logic 440 mayutilize distance information 140, blink information 160, and/or lightinginformation 180 to automatically calculate flash information 460, whichmay automatically remove red eye in an image by detecting a gazedirection of the subject and delaying a firing of a flash until thesubject looks away.

Returning to FIG. 9, red eye may be automatically removed by directinglight from the flash at an oblique angle to the eye (block 920), red eyemay be automatically removed by opening a lens aperture and omitting theflash (block 930), and/or red eye may be automatically removed byreducing a shutter speed and omitting the flash (block 940). Forexample, in one implementation described above in connection with FIG.4, red eye removal logic 440 may utilize distance information 140, blinkinformation 160, and/or lighting information 180 to automaticallycalculate flash information 460, which may automatically remove red eyein an image by redirecting the flash away from an optical axis of userdevice 110 so that light from the flash strikes the eyes at an obliqueangle. Red eye removal logic 440 may utilize distance information 140,blink information 160, and/or lighting information 180 to automaticallycalculate other red eye removal information 480, which may automaticallyremove red eye in an image by capturing the image without a flash, byopening a lens aperture, and by reducing shutter speed.

Process block 855 may include the process blocks depicted in FIG. 10. Asillustrated, process block 855 may include providing for display asuggestion to increase ambient lighting (block 1000), and/or providingfor display a suggestion to move toward or away from the subject (block1010). For example, in one implementation described above in connectionwith FIG. 7, user interface 700 may be provided on display 220 of userdevice 110, and may include manual red eye removal suggestions, such asincrease lighting suggestion 710 and move toward/away from subjectsuggestion 720. Lighting suggestion 710 may provide a visual suggestion(e.g., textual, graphical, textual/graphical, etc.) to increase lightingaround, adjacent to, etc. the subject. If the lighting is increased,user device 110 may detect that lighting is sufficient to omit using aflash, which may eliminate red eye in subject image 510. Movetoward/away from subject 720 may provide a visual suggestion (e.g.,textual, graphical, textual/graphical, etc.) to move user device 110toward or away from the subject. Movement of user device 110 toward oraway from the subject may eliminate red eye in subject image 510.

CONCLUSION

Implementations described herein may predict red eye by detecting eyesof a subject using eye blinks, and may eliminate red eye by increasinglight sensitivity or by adapting a pre-flash length of time based on arisk of red eye. For example, in one implementation, a user device maymeasure a distance from an eye of the subject, may measure a blink ofthe subject's eye, and/or may measure lighting adjacent to the subject.The user device may also calculate automated red eye removal settingsbased on the measured information, may determine manual red eye removalsuggestions based on the measured information, and may provide fordisplay an automatic red eye removal option and/or a manual red eyeremoval option. If the user device receives selection of the automaticred eye removal option, the user device may automatically remove red eyebased on the calculated automated red eye removal settings, and maycapture an image of the subject. If the user device receives selectionof the manual red eye removal option, the user device may provide fordisplay the manual red eye removal suggestions, and may capture theimage of the subject when the manual suggestions remove red eye.

The foregoing description of implementations provides illustration anddescription, but is not intended to be exhaustive or to limit theinvention to the precise form disclosed. Modifications and variationsare possible in light of the above teachings or may be acquired frompractice of the invention.

For example, while series of blocks have been described with regard toFIGS. 8A-10, the order of the blocks may be modified in otherimplementations. Further, non-dependent blocks may be performed inparallel.

Also, the term “user” has been used herein, and is intended to bebroadly interpreted to include user device 110 or a user of user device110.

It should be emphasized that the term “comprises/comprising” when usedin the this specification is taken to specify the presence of statedfeatures, integers, steps or components but does not preclude thepresence or addition of one or more other features, integers, steps,components or groups thereof.

It will be apparent that aspects, as described above, may be implementedin many different forms of software, firmware, and hardware in theimplementations illustrated in the figures. The actual software code orspecialized control hardware used to implement these aspects should notbe construed as limiting. Thus, the operation and behavior of theaspects were described without reference to the specific softwarecode—it being understood that software and control hardware could bedesigned to implement the aspects based on the description herein.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the invention. In fact, many of these features may becombined in ways not specifically recited in the claims and/or disclosedin the specification.

No element, block, or instruction used in the present application shouldbe construed as critical or essential to the invention unless explicitlydescribed as such. Also, as used herein, the article “a” is intended toinclude one or more items. Where only one item is intended, the term“one” or similar language is used. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise.

1. A method, comprising: measuring one or more of a distance from an eye of a subject, a blink of the eye, and lighting adjacent to the subject; calculating automated red eye removal settings based on one or more of the measured distance, blink, and lighting; automatically removing red eye from an image of the subject based on the calculated automated red eye removal settings; capturing the image of the subject; and storing the captured image.
 2. The method of claim 1, further comprising: providing for display an automatic red eye removal option; and receiving selection of the automatic red eye removal option.
 3. The method of claim 1, further comprising: determining manual red eye removal suggestions based on the measured distance, blink, and lighting; providing for display a manual red eye removal option; receiving selection of the manual red eye removal option; providing for display the manual red eye removal suggestions; and capturing the image of the subject when the manual red eye removal suggestions remove red eye.
 4. The method of claim 3, where providing for display the manual red removal suggestions comprises at least one of: providing for display a suggestion to increase ambient lighting; or providing for display a suggestion to move toward or away from the subject.
 5. The method of claim 1, where automatically removing red eye from an image of the subject comprises at least one of: automatically removing red eye from the subject image by increasing or decreasing a length of time of a pre-flash; automatically removing red eye from the subject image by delaying a flash of a device until the subject looks away from the device; automatically removing red eye from the subject image by directing light from the flash at an oblique angle to the eye; automatically removing red eye from the subject image by opening a lens aperture and omitting the flash; or automatically removing red eye from the subject image by reducing a shutter speed and omitting the flash.
 6. A user device, comprising: a distance monitoring device that measures a distance from an eye of a subject; a blink monitoring device that measures a blink of the eye; a lighting monitoring device that measures lighting adjacent to the subject; and processing logic configured to: calculate automated red eye removal settings based on the measured distance, blink, and lighting, automatically remove red eye from an image of the subject based on the calculated automated red eye removal settings, and capture the subject image.
 7. The user device of claim 6, where the distance monitoring device comprises one of: a proximity sensor; a laser distance sensor; a distance sensor using echo location with high frequency sound waves; or an infrared distance sensor.
 8. The user device of claim 6, where the blink monitoring device comprises one of: an eye gazing sensor; an infrared camera and infrared illuminator combination; or a device used to sense, locate, and follow the movement of the eye.
 9. The user device of claim 6, where the blink monitoring device detects the blink as a dual symmetrical difference in images of the subject.
 10. The user device of claim 6, where the lighting monitoring device includes one of: an ambient light sensor; a charge coupled device (CCD) light sensor; or a complementary metal oxide semiconductor (CMOS) light sensor.
 11. The user device of claim 6, where the processing logic is further configured to: provide for display an automatic red eye removal option; and receive selection of the automatic red eye removal option.
 12. The user device of claim 11, where the processing logic is further configured to: provide for display the subject image with red eyes prior to the automatic removal of red eye; provide for display the subject image with non-red eyes after the automatic removal of red eye; provide for display an indication of removal of red eye; and provide for display instructions for capturing the subject image.
 13. The user device of claim 6, where the processing logic is further configured to: determine manual red eye removal suggestions based on the measured distance, blink, and lighting; provide for display a manual red eye removal option; receive selection of the manual red eye removal option; provide for display the manual red eye removal suggestions; and automatically capture the subject image when the manual red eye removal suggestions remove red eye.
 14. The user device of claim 13, where the processing logic is further configured to at least one of: provide for display a suggestion to increase ambient lighting; or provide for display a suggestion to move toward or away from the subject.
 15. The user device of claim 14, where the processing logic is further configured to: provide for display the subject image with red eyes prior to the manual removal of red eye; provide for display the subject image with non-red eyes when the manual red eye removal suggestions remove red eye; provide for display an indication of removal of red eye; and provide for display instructions for capturing the subject image.
 16. The user device of claim 6, where the processing logic is further configured to at least one of: automatically remove red eye from the subject image by increasing or decreasing a length of time of a pre-flash; automatically remove red eye from the subject image by delaying a flash until the subject looks away from the user device; automatically remove red eye from the subject image by directing light from the flash at an oblique angle to the eye; automatically remove red eye from the subject image by opening a lens aperture and omitting the flash; or automatically remove red eye from the subject image by reducing a shutter speed and omitting the flash.
 17. The user device of claim 6, where the user device comprises one of: a mobile communication device; a laptop; a personal computer; a camera; a video camera; binoculars with a camera; or a telescope with a camera.
 18. A user device, comprising: one or more devices configured to: measure a distance from an eye of a subject whose image is to be captured, measure a blink of the eye, and measure lighting adjacent to the subject; and processing logic configured to: provide for display an automatic red eye removal option and a manual red eye removal option, receive selection of the automatic red eye removal option or the manual red eye removal option, calculate automated red eye removal settings based on the measured distance, blink, and lighting when the automatic red eye removal option is selected, automatically remove red eye from an image of the subject based on the calculated automated red eye removal settings when the automatic red eye removal option is selected, determine manual red eye removal suggestions based on the measured distance, blink, and lighting when the manual red eye removal option is selected, provide for display the manual red eye removal suggestions when the manual red eye removal option is selected, and enable the subject image to be captured when red eye is removed.
 19. The user device of claim 18, where the processing logic is further configured to: provide for display the subject image with red eyes prior to the removal of red eye; provide for display the subject image with non-red eyes when red eye is removed; provide for display an indication of removal of red eye; and provide for display instructions for capturing the subject image.
 20. A system, comprising: means for measuring a distance from an eye of a subject; means for measuring a blink of the eye; means for measuring lighting adjacent to the subject; means for calculating automated red eye removal settings based on the measured distance, blink, and lighting; means for automatically removing red eye from an image of the subject based on the calculated automated red eye removal settings; means for capturing the image of the subject; and means for storing the captured subject image. 